Imaging device and imaging method

ABSTRACT

At a controller  5,  pixel values that a CCD  3  outputs are evaluated. On the basis of the evaluation result, exposure times with respect to a light receiving surface of the CCD  3  in a shutter  2  constituted with, e.g., DMD (Digital Micromirror Device), etc. are set in pixel units. Further, image pick-up of object is carried out by exposure times set in pixel units in that way. As a result, even in the case of object of high contrast, picture image of which detail is not injured can be obtained.

TECHNICAL FIELD

[0001] This invention relates to an image pick-up apparatus and an imagepick-up method, a program and a program recording medium, a datastructure and a data recording medium, and an image pick-up controlapparatus, and more particularly to an image pick-up apparatus and animage pick-up method, a program and a program recording medium, a datastructure and a data recording medium and an image pick-up controlapparatus adapted so as to have ability to obtain picture image ofobject of high contrast without losing its detail, e.g., in digitalvideo camera, etc.

BACKGROUND ART

[0002] For example, in digital video cameras, light from object isconverged by lens onto the light receiving surface of photo-electricconversion device such as CCD (Charge Coupled Device), etc., at whichphoto-electric conversion is carried out so that picture data which iselectric signal is provided.

[0003] In digital video cameras, light from lens is incident upon CCDthrough shutter which controls exposure. Accordingly, when shutter speedis low, i.e., exposure time is long, many electric charges are chargedin CCD, resulting in the so-called over exposure. Further, picture imageobtained at the time of over exposure results in the so-called whiteoverexposure picture image. On the other hand, when exposure time isshort, quantity of electric charges charged in CCD becomes lesser. As aresult, the so-called under exposure takes place, and picture imageobtained results in black underexposure picture image. In order toprevent such white overexposure or black underexposure, it is necessaryto set exposure time so that the area from the portion where pictureimage is the most bright up to the portion where it is the most dark hassuitable brightness.

[0004] Meanwhile, in conventional digital video cameras, photographingis carried out at the same exposure time with respect to the entirety ofthe light receiving surface of CCD. Accordingly, in the case whereobject of high contrast is photographed (imaged), the bright portionresults in white overexposure portion, or the dark portion results inblack underexposure portion, resulting in the problem that detail ofobject is lost in the picture image thus obtained.

[0005] It is to be noted that while camera with liquid crystal shutteris disclosed in, e.g., the Japanese Patent Application No. 28796/1994,this camera cannot control exposure in pixel units of CCD, but can onlyset the same exposure time with respect to the entirety of the lightreceiving surface of CCD.

DISCLOSURE OF THE INVENTION

[0006] This invention has been made in view of such circumstances and anobject of this invention is to provide an image pick-up apparatus, animage pick-up method, a program, a program recording medium, a datastructure, a data recording medium, and an image pick-up controlapparatus which can pick image of up even object of high contrastwithout losing its detail.

[0007] An image pick-up apparatus according to this invention comprisesimage pick-up means having a light receiving surface for receiving lightfrom object to carry out photo-electric conversion and adapted to outputpixel value obtained as the result of the photo-electric conversion,evaluating means for evaluating pixel value, and control means forcontrolling, in pixel units, exposure time with respect to the lightreceiving surface on the basis of evaluation result by the evaluationmeans.

[0008] An image pick-up method according to this invention includes anevaluation step of evaluating pixel value acquired from an image pick-upunit having a light receiving surface for receiving light from object tocarry out photo-electric conversion and adapted to output pixel valueobtained as the result of the photo-electric conversion, and a controlstep of controlling, in pixel units, exposure time with respect to thelight receiving surface on the basis of evaluation result by theevaluation step.

[0009] A program according to this invention includes an evaluation stepof evaluating pixel value acquired from an image pick-up unit having alight receiving surface for receiving light from object to carry outphoto-electric conversion and adapted to output pixel value obtained asthe result of the photo-electric conversion, and a control step ofcontrolling, in pixel units, exposure time with respect to the lightreceiving surface on the basis of evaluation result by the evaluationstep.

[0010] A program recording medium according to this invention is adaptedso that there is recorded a program including an evaluation step ofevaluating pixel value acquired from an image pick-up unit having alight receiving surface for receiving light from object to carry outphoto-electric conversion and adapted to output pixel value obtained asthe result of the photo-electric conversion, and a control step ofcontrolling, in pixel units, exposure time with respect to the lightreceiving surface on the basis of evaluation result by the evaluationstep.

[0011] A data structure according to this invention is adapted so thatplural pixel values that an image pick-up unit for picking up image ofobject outputs and exposure times every pixel values used in the imagepick-up unit in order to obtain respective plural pixel values arecaused to correspond to each other.

[0012] A data recording medium according to this invention is adapted sothat plural pixel values that an image pick-up unit for picking up imageof object outputs and exposure times every respective pixels used in theimage pick-up unit in order to obtain respective plural pixel values arerecorded in such a manner that they caused to correspond to each other.

[0013] An image pick-up control apparatus according to this inventioncomprises an evaluating unit for evaluating pixel value, and controlmeans for outputting, to an image pick-up unit, a control signal forcontrolling, in predetermined surface units, exposure time with respectto a light receiving surface on the basis of evaluation result by theevaluating unit.

[0014] An image pick-up apparatus according to this invention comprisesimage pick-up means having a light receiving surface for receiving lightfrom object to carry out photo-electric conversion and adapted to outputpixel value obtained as the result of the photo-electric conversion,control means for controlling plural exposure times with respect to thelight receiving surface, and selector means for selecting one pixelvalue from pixel values corresponding to respective plural exposuretimes of the respective pixel positions which are obtained by picking upimage of object by plural exposure times on the basis of control of thecontrol means in the image pick-up means.

[0015] An image pick-up method according to this invention includes acontrol step of controlling plural exposure times with respect to alight receiving surface at an image pick-up unit having the lightreceiving surface for receiving light from object to carry outphoto-electric conversion and adapted to output pixel value obtained asthe results of the photo-electric conversion, and a selection step ofselecting one pixel value from pixel values corresponding to respectiveplural exposure times of respective pixel positions which are obtainedby picking up image of object by plural exposure times on the basis ofcontrol at the control step in the image pick-up unit.

[0016] A program according to this invention includes an evaluation stepof evaluating pixel values corresponding to respective plural exposuretimes of respective pixel positions which are obtained by controllingplural exposure times with respect to a light receiving surface at animage pick-up unit having the light receiving surface for receivinglight from object to carry out photo-electric conversion and adapted tooutput pixel value obtained as the result of the photo-electricconversion, and a selection step of selecting one pixel value from pixelvalues corresponding to respective plural exposure times on the basis ofevaluation result by the evaluation step.

[0017] A program recording medium according to this invention is adaptedso that there is recorded a program including an evaluation step ofevaluating pixel values corresponding to respective plural exposuretimes of respective pixel positions which are obtained by controllingplural exposure times with respect to a light receiving surface at animage pick-up unit having the light receiving surface for receivinglight from object to carry out photo-electric conversion and adapted tooutput pixel value obtained as the result of the photo-electricconversion, and a selection step of selecting one pixel value from pixelvalues corresponding to respective plural exposure times on the basis ofevaluation result by the evaluation step.

[0018] In this invention, pixel value acquired from image pick-up meanshaving a light receiving surface for receiving light from object tocarry out photo-electric conversion and adapted to output pixel valueobtained as the result of the photo-electric conversion is evaluated,and exposure time with respect to the light receiving surface iscontrolled in pixel units on the basis of the evaluation result.

[0019] Moreover, in this invention, plural pixel values that an imagepick-up apparatus (unit) for picking up image of object outputs andexposure times every respective pixels used in the image pick-up unit inorder to obtain respective plural pixel values are caused to correspondto each other.

[0020] Further, in this invention, pixel value is evaluated, and acontrol signal for controlling, in predetermined surface units, exposuretime with respect to the light receiving surface on the basis of itsevaluation result is outputted to the image pick-up unit.

[0021] Further, in this invention, plural exposure times with respect tothe light receiving surface in image pick-up means having lightreceiving surface for receiving light from object and adapted to outputpixel value obtained as the result of the photoelectric conversion arecontrolled, and one pixel value is selected from pixel valuescorresponding to respective plural exposure times of respective pixelpositions which are obtained by picking up image of object by pluralexposure times on the basis of the control.

[0022] In addition, in this invention, pixel values corresponding torespective plural exposure times of respective pixel positions which areobtained by controlling plural exposure times with respect to lightreceiving surface at image pick-up unit having the light receivingsurface for receiving light from object to carry out photo-electricconversion and adapted to output pixel value obtained as the result ofthe photo-electric conversion are evaluated, and one pixel value isselected from pixel values corresponding to respective plural exposuretimes on the basis of the evaluation result.

BRIEF DESCRIPTION OF THE DRAWINGS

[0023]FIG. 1 is a block diagram showing an example of configuration of adigital video camera system to which this invention is applied.

[0024]FIG. 2 is a view showing an example of configuration of a firstembodiment of digital video camera 101 to which this invention isapplied.

[0025]FIG. 3 is a block diagram showing an example of configuration of asecond embodiment of digital video camera 101 to which this invention isapplied.

[0026]FIG. 4 is a block diagram showing an example of configuration ofcontroller 5 of FIGS. 2 and 3.

[0027]FIG. 5 is a block diagram showing an example of configuration ofpicture image evaluating unit 11 of FIG. 4.

[0028]FIG. 6 is a block diagram showing an example of configuration ofevaluating section 23 of FIG. 5.

[0029]FIG. 7 is a flowchart for explaining operation of digital videocamera 101 of FIG. 4 (FIGS. 2 and 3).

[0030]FIGS. 8A, 8B, 8C are views showing storage contents of memories 6and 25.

[0031]FIG. 9 is a block diagram showing an example of configuration ofdisplay unit 102 of FIG. 1.

[0032]FIG. 10 is a flowchart for explaining processing of display unit102 of FIG. 9.

[0033]FIG. 11 is a block diagram showing an example of configuration ofa third embodiment of digital video camera 101 to which this inventionis applied.

[0034]FIG. 12 is a block diagram showing an example of configuration ofcontroller 33 of FIG. 11.

[0035]FIG. 13 is a flowchart for explaining operation of digital videocamera 101 of FIG. 11.

[0036]FIG. 14 is a view showing the relationship between pixel value andexposure time.

[0037]FIG. 15 is a view for explaining correction of pixel value usingapproximate curve which approximates the relationship between pixelvalue and exposure time.

[0038]FIG. 16 is a view showing pixel values obtained by plural exposuretimes.

[0039]FIG. 17 is a view showing corrected pixel value.

[0040]FIG. 18 is a view showing picture image constituted with correctedpixel value obtained by correcting pixel value closest to referencepixel value.

[0041]FIG. 19 is a block diagram showing an example of configuration ofan embodiment of a computer to which this invention is applied.

BEST MODE FOR CARRYING OUT THE INVENTION

[0042]FIG. 1 shows an example of the configuration of an embodiment of adigital video camera system to which this invention is applied. Adigital video camera 101 picks up image of a predetermined object tooutput picture information including picture data. This pictureinformation is recorded with respect to a recording medium 103 comprisedof, e.g., semiconductor memory, magneto-optical disc, magnetic disc,optical disc, magnetic tape or phase change disc, etc., or is caused toundergo transmission through a transmission medium 104 comprised of,e.g., ground wave wireless circuit, satellite circuit, CATV (CableTelevision) network, internet, public circuit, or bus, etc., and isoffered to a display unit 102. The display unit 102 receives pictureinformation provided through the recording medium 103 or thetransmission medium 104 to display corresponding picture image on thebasis of the received picture information. It is to be noted that thedigital video camera 101, the display unit 102 and the recording medium103 or the transmission medium 104 may be constituted as a singleapparatus.

[0043]FIG. 2 shows an example of the configuration of the firstembodiment of the digital video camera 101 of FIG. 1. Light from objectis incident upon a lens 1, and the lens 1 converges that light onto thelight receiving surface of a CCD 3 through a shutter 2.

[0044] The shutter 2 is controlled by a controller 5, and serves toreflect light from the lens 1, e.g., in pixel units constituting thelight receiving surface of the CCD 3 to thereby control exposures withrespect to respective pixels of the CCD 3. Namely, in FIG. 2, theshutter 2 is constituted with, e.g., DMD (Digital Micromirror Device) inwhich a large number of very small reflection mirrors are formed onsemiconductor substrate, and respective mirrors are rotated inaccordance with control from the controller 5 so that reflectiondirection of light incident thereupon can be changed in those mirrorunits.

[0045] In this example, respective mirrors constituting DMD correspondto respective pixels constituting the CCD 3. Accordingly, the respectivemirrors are rotated to thereby change reflection direction of light fromthe respective mirrors to the respective pixels of CCD 3 so thatincidence of light upon corresponding pixels can be turned ON/OFF. It isto be noted that DMD is disclosed in, e.g., the Japanese PatentApplication No. 73952/1995 (Priority No. US221739, Priority Date: Mar.31, 1994).

[0046] The CCD 3 receives light from the shutter 2 in respective pixelsconstituting that light receiving surface to thereby charge electriccharges corresponding to that light quantity. Further, the CCD 3 carriesout the so-called bucket brigade of electric charges charged inrespective pixels (electric charges integrated by integral effect) tooutput electric signal of corresponding voltage level to an A/D(Analog/Digital) converter 4. While CCD is used as photo-electricconversion device for receiving light from the shutter 2 to carry outphoto-electric conversion in this example, BBD (Bucket Brigade Device),etc. may be used in addition to CCD.

[0047] The A/D converter 4 carries out sampling of electric signal fromthe CCD 3 at timing corresponding to pixels to quantize it to therebydeliver pixel values of respective pixels constituting digital picturedata to the controller 5. In this example, the A/D converter 4 isassumed to output, e.g., pixel value of 8 bits.

[0048] The controller 5 evaluates pixel values of respective pixelsdelivered through the A/D converter 4 from the CCD 3. Further, thecontroller 5 sets, in respective pixel units, exposure time by shutter 2on the basis of the evaluation result to control the shutter 2. Further,the controller 5 corrects, as occasion demands, pixel values ofrespective pixels delivered through the A/D converter 4 from the CCD 3on the basis of exposure time set when those pixel values are obtainedto output, as picture information, picture data of, e.g., 1 frame (or 1field) units consisting of corrected pixel values. In addition, thecontroller 5 outputs, as picture information, pixel value and exposuretime set when corresponding pixel value is obtained. Picture informationthat the controller 5 outputs are received by an I/F (Interface) 7.

[0049] A memory 6 temporarily stores data necessary for processing ofthe controller 5. The I/F 7 delivers picture information from thecontroller 5 to a recording section 8 or a communication section 9 inaccordance with, e.g., instruction from user, etc. The recording section8 records picture information from the I/F 7 onto a recording medium103. The communication section 9 transmits picture information from theI/F 7 through a transmission medium 104.

[0050]FIG. 3 shows an example of configuration of a second embodiment ofthe digital video camera 101 of FIG. 1. It is to be noted that the samereference numerals are respectively attached to portions correspondingto the case in FIG. 2 in the figure and its explanation will be omittedas occasion demands.

[0051] Namely, the digital video camera 101 of FIG. 3 is constitutedessentially similarly to the digital video camera 101 of FIG. 2. In FIG.3, the shutter 2 is constituted with liquid crystal panel (liquidcrystal shutter).

[0052] The shutter 2 constituted with the liquid crystal panel iscontrolled by the controller 5 to allow light from the lens 1 to betransmitted in pixel units constituting, e.g., light receiving surfaceof the CCD 3 to thereby control exposures with respect to respectivepixels of the CCD 3.

[0053] Namely, in FIG. 3, direction of liquid crystal moleculesconstituting liquid crystal panel as the shutter 2 is changed in unitscorresponding to pixel in accordance with control of the controller 5 sothat transmission of light in that unit is limited. Thus, incidence oflight upon corresponding pixel can be turned ON/OFF. While the liquidcrystal panel of the transmission type is used as the shutter 2 in thisexample, liquid crystal panel of the reflection type may be used inaddition to the above.

[0054]FIG. 4 shows an example of the configuration of the controller 5of FIGS. 2 and 3. The controller 5 is constituted with a pictureevaluating section 11 and a shutter control section 12.

[0055] Pixel value delivered from the CCD 3 to the controller 5 throughthe A/D converter 4 is received at the picture evaluating section 11.The picture evaluating section 11 implements necessary processing topixel value delivered thereto to constitute picture data of one frame tooutput such picture data. Further, the picture evaluating section 11evaluates pixel value delivered thereto to set, in pixel units, exposuretime by the shutter 2 on the basis of the evaluation result.

[0056] The shutter control section 12 controls the shutter 2 inaccordance with exposure time every pixel set at the picture evaluatingsection 11.

[0057]FIG. 5 shows an example of the configuration of the pictureevaluating section 11 of FIG. 4. Pixel value delivered from the CCD 3 tothe controller 5 through the A/D converter 4 is received at a buffer 21,and the buffer 21 temporarily stores its pixel value.

[0058] A pixel value correcting section 22 reads out pixel value storedin the buffer 21 and reads out, from a memory 25, exposure time withrespect to pixel when corresponding pixel value is obtained to deliverthem to the memory 6 in a manner such that they are caused to correspondto each other to allow the memory 6 to store them. Further, the pixelvalue correcting section 22 is operative so that when, e.g., set ofpixel value and exposure time corresponding to one frame is stored inthe memory 6, it reads out the set of pixel value and exposure time tocorrect pixel value on the basis of exposure time to output picture dataof one frame constituted by the corrected pixel value.

[0059] An evaluating section 23 evaluates pixel value stored in thebuffer 21 to deliver its evaluation result to an exposure timedetermining section 24. The exposure time determining section 24 setsexposure time with respect to pixel of pixel value stored in the buffer21 on the basis of evaluation result from the evaluating section 23.Namely, the evaluating section 23 evaluates pixel value stored in thebuffer 21 to obtain evaluation result of whether pixel value is aboveupper limit or below lower limit, or movement quantity of object, etc.to deliver its evaluation result to the exposure time determiningsection 24. The exposure time determining section 24 is adapted so thatwhen, e.g., pixel value is value above a predetermined value and thereresults white overexposure state, it sets exposure time with respect tocorresponding pixel to shorter value. Moreover, the exposure timedetermining section 24 is adapted so that when, e.g., pixel value isvalue below predetermined value and there results black underexposurestate, it sets exposure time with respect to corresponding pixel tolonger value. Further, the exposure time determining section 24 isadapted so that when, e.g., movement quantity of object is large andthere results unintentional movement (movement blurring), it setsexposure time with respect to corresponding pixel to shorter value. Inaddition, the exposure time determinating section 24 is adapted so thatwhen, e.g., movement quantity of object is small (no movement quantity)and there results no unintentional movement (blurring), exposure timewith respect to corresponding pixel is caused to remain at presentvalue.

[0060] Further, the exposure time determinating section 24 deliversexposure time set with respect to pixel to a memory 25. In thisembodiment, for simplifying explanation, it is assumed that there ishardly movement of object itself. Accordingly, it is assumed thatunintentional movement (movement blurring) referred to here results fromthe so-called hand movement of user who carries out photographing. It isto be noted that this invention can be applied to the case where objecthaving large movement is photographed (imaged).

[0061] The memory 25 stores (overwrites) exposure times with respect torespective pixels from the exposure time determinating section 24 ataddress of corresponding position. Exposure times with respect torespective pixels stored in the memory 25 are delivered to a shuttercontrol section 12, and the shutter control section 12 controls theshutter 2 in accordance with exposure times every respective pixels.Thus, incident time of light upon the CCD 3 through the shutter 2 iscontrolled every pixel. In this time, the exposure time and the shutterspeed are synonym. It is to be noted that the fact that the exposuretime is long corresponds to the fact that the shutter speed is low andthe fact that the exposure time is short corresponds to the fact thatthe shutter speed is high. While explanation will be given by usingexposure time in the following description, it is possible as a matterof cause that explanation will be given by using shutter speed.

[0062]FIG. 6 shows an example of the configuration of evaluating section23 of FIG. 5. A read-out section 51 reads out pixel value of pixelstored in the buffer 21 (FIG. 5) to deliver such pixel value to a buffer52 to allow the buffer 52 to store it. Further, the read-out section 51allows pixels of which pixel values have been read out from the bufferto be remarked pixels in order to deliver pixel value of that remarkedpixel (remarked pixel value) to a movement judging section 53 and apixel value judging section 54.

[0063] The buffer 52 has memory capacity capable of storing, e.g., pixelvalues corresponding to plural frames, and stores in order pixel valuesdelivered from the read-out section 51. In this example, the buffer 52is adapted to store new pixel value, e.g., in such a form to overwritethe new pixel value onto the oldest pixel value after it stores pixelvalue by its memory capacity.

[0064] The movement judging section 53 receives pixel value of remarkedpixel from the read-out section 51 to read out, from the buffer 52,e.g., 3×3 pixels in frame preceding by one frame of remarked frame(hereinafter referred to as preceding frame as occasion demands)corresponding to 3×3 pixels with remarked pixel being as center at frameof the remarked pixel (hereinafter referred to as remarked frame asoccasion demands). Further, the movement judging section 53 calculatesabsolutes of differences between respective pixel values of 3×3 pixelsof remarked frame and respective pixel values of corresponding 3×3pixels at the preceding frame to further calculate its sum total(hereinafter referred to as difference absolute value sum as occasiondemands). Further, the movement judging section 53 judges magnitude ofmovement of remarked pixel on the basis of the difference absolute valuesum to deliver its judgment result to the exposure time determiningsection 24 (FIG. 5) as evaluation result of remarked pixel. Namely, themovement determining section 53 is operative so that when differenceabsolute value is large, it delivers output to the effect that movementis large to the exposure time determining section 24 as evaluationresult, while when difference absolute value is small, it deliversoutput to the effect that movement is small to the exposure timedetermining section 24 as evaluation result. It is to be noted that, atthe movement judging section 53, in addition to the above, e.g., 3×3pixels of the above-described remarked frame may be used to carry outblock matching between current frame and forward (preceding) frame tojudge magnitude of movement of the remarked pixel on the basis of movingvector obtained as the result of the block matching.

[0065] The pixel value judging section 54 judges whether or not pixelvalue of remarked pixel is value within a predetermined range prescribedby a predetermined lower limit value and a predetermined upper limitvalue to deliver its judgment result to the exposure time determiningsection 24 as evaluation result of the remarked pixel.

[0066] Namely, in the case where pixel value that the A/D converter 4(FIGS. 2 to 4) outputs is represented by 8 bits corresponding to, e.g.,the range of 0 to 255, the pixel value judging section 54 allows, e.g.,value where there results black underexposure state (e.g., 10 or less)to be lower limit value and allows value where there results whiteoverexposure state (e.g., 250, etc.) to be upper limit value, thus tojudge whether or not pixel value of remarked pixel exists within therange from lower limit value to upper limit value. Further, the pixelvalue judging section 54 is operative so that in the case where pixelvalue of remarked pixel is upper limit or more (or value larger thanupper value), in the case where it is lower limit value or less (orvalue less than the lower limit value) and in the case where it islarger than the lower limit value and is smaller than the upper limitvalue, the pixel value judging section 54 delivers outputs to the effectof these cases to the exposure time determining section 24 as evaluationresult of remarked pixel.

[0067] The exposure time determining section 24 of FIG. 5 receivesevaluation result of remarked pixel as described above from the movementjudging section 53 and the pixel value judging section 54 to setexposure time in a manner as described above on the basis of theevaluation result.

[0068] The operation of the digital video camera of FIG. 4 (FIGS. 2 and3) will now be described with reference to the flowchart of FIG. 7.

[0069] Initially, at step S1, the exposure time determining section 24of the controller 5 (FIG. 5) sets exposure times of default with respectto respective pixels to transmit such exposure times to the memory 25 toallow the memory 25 to store them at corresponding addresses. In thisexample, it is assumed that the same exposure time is set with respectto all pixels, e.g., as exposure time of default. It is to be notedthat, as exposure time of default, e.g., exposure times of respectivepixels at the time of completion of the last photographing, etc. may beset. In addition, exposure time of default may be set by user, forexample. In this case, FIG. 8A shows memory content of memory 25. Asdescribed above, at the memory 25, exposure times with respect tocorresponding pixels are stored at addresses corresponding to pixelpositions of respective pixels. The shutter control section 12 controlsthe shutter 2 in accordance with exposure times every pixels stored atthe memory 25. Thus, while incident times of light upon CCD 3 throughthe shutter 2 are controlled every pixel, electric charges are chargedat respective pixels of the CCD 3.

[0070] Further, when there results read-out start timing of pixel valueconstituting one frame, its read-out operation is started from the CCD 3at step S2. Pixel value which has been read out from the CCD 3 isdelivered to a buffer 21 of the controller 5 (FIG. 5) through the A/Dconverter 4, and is stored thereat.

[0071] Pixel value stored in the buffer 21 is read out by pixel valuecollecting section 22 at step S3. Further, at the step S3, the pixelvalue collecting section 22 reads out exposure time stored at address ofthe memory 25 corresponding to remarked pixel with pixels of pixelvalues which have been read out from the buffer 21 being as remarkedpixel in order, i.e., exposure time used for the purpose of obtainingpixel value of remarked pixel to deliver such exposure time to thememory 6 in such a manner that it is caused to correspond to pixel valueof remarked pixel to allow the memory 6 to store the exposure time.Thus, in the case where exposure time as shown in FIG. 8A, for example,is stored at the memory 25, exposure time of FIG. 8A and pixel valueobtained by that exposure time are stored at the memory 6 in such amanner that they are caused to correspond to each other as shown in FIG.8B.

[0072] Further, processing proceeds to step S4. The evaluating section23 (FIG. 6) evaluates pixel value of remarked pixel stored at the buffer21 in a manner as described above at the movement judging section 53 andthe pixel value judging section 54 to deliver its evaluation result tothe exposure time determining section 24. The exposure time determiningsection 24 sets, for a second time, exposure time of remarked pixel toreasonable value as described above on the basis of evaluation resultfrom the evaluating section 23 at step S5. Further, the exposure timedetermining section 24 delivers the exposure time which has been set fora second time to the memory 25 to allow the exposure time to be stored(overwritten) in address corresponding to remarked pixel. Namely, e.g.,in FIG. 8B, pixel value “250” is value of the above-described upperlimit “250” or more. Accordingly, it is considered that there resultsstate of white overexposure state. For this reason, the exposure timedetermining section 24 sets for a second time exposure time of pixel ofpixel value “250” (exposure time of address n+1 of memory 25) “{fraction(1/100)} sec.” to shorter “{fraction (1/120)} sec.” to allow the memory25 to store it as shown in FIG. 8C.

[0073] Thereafter, processing proceeds to step S6. Whether or notread-out operation of all pixel values Constituting one frame from thebuffer 21 is completed is judged by, e.g., control unit (not shown),etc. In the case where it is judged that read-out operation is not yetcompleted, processing proceeds to step S7, at which pixel value of thenext pixel is acquired from the buffer 21. Thus, processing returns tothe step S3. Further, processing at step S3 and at steps subsequentthereto will be repeated with pixel of that pixel value being newly asremarked pixel.

[0074] On the other hand, in the case where it is judged at the step S6that read-out operation of all pixel values constituting one frame iscompleted, i.e., in the case where pixel values of all pixelsconstituting one frame and exposure times caused to correspond theretoare stored, processing proceeds to step S8. Thus, the pixel valuecorrecting section 22 (FIG. 5) reads out respective pixel values fromthe memory 6 to correct respective pixel values on the basis of exposuretimes caused to correspond to those pixel values to output picture dataof one frame constituted with corrected pixel value.

[0075] Namely, since respective pixel values constituting one frame arenot pixel values obtained by the same exposure time, when such pixelvalues are used as they are to constitute picture of one frame, thereresults a picture image in which brightness is sparse or thin. In viewof this, the pixel value correcting section 22 corrects respective pixelvalues on the basis of exposure time to thereby constitute a pictureimage such that there is feeling of unity in brightness and all pixelsare photographed by the same exposure time.

[0076] In more practical sense, for the purpose of simplifyingexplanation, e.g., exposure time and pixel value are assumed to beproportional relationship. In the pixel value correcting section 22,when, e.g., the shortest exposure time 1/S_(BASE) [sec.] of exposuretimes stored in the memory 6 is assumed as reference and set of pixelvalue p exposure time 1/S [sec.] and exposure time is expressed as (p,1/S), in the case where combination of pixel value and exposure timestored in the memory 6 is, e.g., (255, {fraction (1/10)}), (200, ⅕),(150, {fraction (1/20)}), (100, {fraction (1/100)}), since the shortestexposure time 1/S_(BASE) is {fraction (1/100)} sec., pixel value “255”where exposure time is {fraction (1/10)} sec. is corrected to “25.5”(=255×{fraction (10/100)}), pixel value “200” where exposure time is ⅕sec. is corrected to “10” (=200×5/100), pixel value “150” where exposuretime is {fraction (1/20)} sec. is corrected to “30” (=150×20/100), andpixel value “100” where exposure time is {fraction (1/100)} sec. iscorrected to “100” (=100×100/100). It is to be noted that pixel value“100” of the shortest exposure time “1/100 sec.” serving as reference isnot changed before and after correction, it is not necessarily requiredto make correction.

[0077] While, in the above-described case, the shortest exposure time1/S_(BASE) is caused to be as reference to make a correction such thatpixel values to which respective exposure times 1/S are caused tocorrespond are changed to multiple of S/S_(BASE), such an approach maybe also employed in addition to the above to, e.g., use the longestexposure time 1/S_(BASE)′ [sec.] of exposure times stored in the memory6 as reference to make a correction such that pixel values to whichrespective exposure times 1/S are caused to correspond are changed tomultiple of S/S_(BASE)′. In this case, with respect to sets of theabove-described pixel value and exposure time (255, {fraction (1/10)}),(200, ⅕), (150, {fraction (1/20)}), (100, {fraction (1/100)}), becausethe longest exposure time 1/S_(BASE)′ is ⅕, pixel value “255” whereexposure time is {fraction (1/10)} sec. is corrected to “510”(=255×10/5), pixel value “200” where exposure time is ⅕ sec. iscorrected to “200” (=200×5/5), pixel value “150” where exposure time is{fraction (1/20)} sec. is corrected to “600” (=150×20/5) and pixel value“100” where exposure time is {fraction (1/100)} sec. is corrected to“2000” (=100×100/5).

[0078] It is to be noted that the pixel value correcting section 22 cancarry out correction of pixel values with exposure times except for theshortest exposure time and the longest exposure time being as referencein addition to the above.

[0079] From the foregoing description, when pixel value that the A/Dconverter 4 outputs is assumed to be M bits, there are the cases wherepixel value constituting corrected picture that the pixel valuecorrecting section 22 outputs may be the number of bits above M bits.Namely, while pixel values that the A/D converter 4 outputs are 8 bits,pixel values constituting corrected picture that the pixel valuecorrecting section 22 outputs is the number of bits above 8 bits. As aresult, from the pixel value correcting section 22, even if object hashigh contrast, picture image in which its contrast is sufficientlyrepresented, i.e., picture image in which dynamic range is caused to belarger than the number of output bits of the A/D converter 4.

[0080] While the shortest exposure time is caused to be reference inthis example, exposure time serving as reference may be arbitrary value.Namely, exposure time serving as reference may be exposure times exceptfor the shortest exposure time stored in the memory 6, or may beexposure time which is not stored in the memory 6. Accordingly,correction of pixel value can be carried out, e.g., with {fraction(1/1)} sec. being as reference. In this case, pixel value obtained bymultiplying pixel value p by inverse number S of exposure time 1/Scaused to correspond thereto results in corrected pixel value.

[0081] It is to be noted that even if correction is made with anyexposure time being as reference value, dynamic range of picturesimilarly becomes large.

[0082] When picture data consisting of corrected pixel value as statedabove is outputted at step S8, processing proceeds to step S9. As theresult of the fact that processing of steps S4 to S7 are repeated,exposure times every respective pixels stored in the memory 25 aretransmitted to shutter control section 12. Thus, processing returns tothe step S2. At times subsequent thereto, with respect to the nextframe, similar processing is repeated. Accordingly, with respect to thenext frame, imaging of picture is carried out by exposure times everyrespective pixels stored in the memory 25.

[0083] As described above, such an approach is employed to evaluatepixel value that the CCD 3 outputs to set, in pixel units, exposure timewith respect to the light receiving surface of the CCD 3 by the shutter2 on the basis of its evaluation result to carry out image pick-upoperation of object. Accordingly, even if there is employed object ofhigh contrast, it is possible to obtain picture of which detail is notinjured.

[0084] Moreover, in general, the dynamic range of CCD is not so broad,but exposure times are controlled every pixels as described above,thereby making it possible to provide the effect similar to the casewhere the dynamic range of the CCD 3 (or the A/D converter 4) isbroadened.

[0085] While, in the above-described case, such an approach is employedto collect respective pixel values stored in the memory 6 on the basisof exposure times caused to correspond to those pixel values and tooutput them, respective pixel values stored in the memory 6 may beoutputted along with exposure times caused to correspond to those pixelvalues as they are to record such pixel values onto recording medium 103or to carry out transmission thereof through transmission medium 104.

[0086]FIG. 9 shows an example of the configuration of the displaysection 102 of FIG. 1. A read-out section 61 reads out (reproduces),from recording medium 103, picture information recorded thereon(corrected pixel value, or pixel value and exposure time) to deliver itto an I/F 63. A communication section 62 receives picture informationtransmitted through the transmission medium 104 to deliver it to the I/F63. The I/F 63 receives picture information delivered from the read-outsection 61 or the communication section 62 to deliver it to a displaycontrol section 64. The display control section 64 is composed of abuffer 65, a pixel value normalizing section 66, and a driver 67. Thebuffer 65 receives picture information delivered from the I/F 63 totemporarily store it in units of one frame, for example. The pixel valuenormalizing section 66 reads out picture information stored in thebuffer 65 to normalize that picture information on the basis of displayaccuracy of a display section 68.

[0087] Namely, the pixel value normalizing section 66 recognizes,through the driver 67, display accuracy of the display section 68, i.e.,what bit of pixel value the display section 68 can display. In thisexample, display accuracy of the display section is assumed to be Kbits. Further, the pixel value normalizing section 66 is operative sothat in the case where picture information corresponding to one framestored in the buffer 65 is corrected pixel value (hereinafter referredto as corrected pixel value, it detects the maximum value thereof. Now,in the case where this maximum pixel value is expressed as K′ (>K), thepixel value normalizing section 66 rounds down lower order K′-K bits ofrespective corrected pixel value stored in the buffer 65 to normalizepixel value so that K bits are provided. Moreover, the pixel valuenormalizing section 66 is operative so that in the case where pictureinformation stored in the buffer 65 is pixel value p and exposure time1/S, e.g., it multiplies pixel value p by inverse number S of exposuretime 1/S similarly to the case in the pixel value correcting section 22of FIG. 5 to thereby correct pixel value p to determine corrected pixelvalue pxS. Further, similarly to the above-described case, the pixelvalue normalizing section 66 rounds down lower order K′-K bits ofrespective corrected pixel value to normalize pixel value so that K bitsare provided. The pixel value normalizing section 66 normalizes pixelvalue in a manner as stated above to deliver pixel value afternormalization (hereinafter referred to as normalized pixel value asoccasion demands) to the driver 67.

[0088] The driver 67 communicates with the display section 68 to therebyrecognize its display accuracy. Alternatively, the driver 67 recognizesin advance display accuracy of the display section 68. Further, thedriver 67 delivers display accuracy of the display section 68 to thepixel value normalizing section 66 in accordance with request of thepixel value normalizing section 66. In addition, the driver 67 drivesthe display section 68 in accordance with normalized pixel valuedelivered from the pixel value normalizing section 66 to thereby displaypicture image on the display section 68. The display section 68 iscomprised of, e.g., CRT or liquid crystal display, etc., and serves todisplay picture image in accordance with control from the driver 67.

[0089] The display unit 102 of FIG. 9 will now be described withreference to the flowchart of FIG. 10. The I/F 63 receives pictureinformation delivered from the readout section 61 or the communicationsection 62 to deliver it in order one by one frame to the buffer 65 ofthe display control section 64 to allow the buffer 65 to store it.

[0090] Further, at step S11, the pixel value normalizing section 66recognizes display accuracy of the display section 68 through the driver67. It is to be noted that since the display accuracy indicates that thedisplay section 68 can display the number of bits of pixel value whichcan be displayed as described above, the display accuracy may be dynamicrange which is difference between the maximum value and the minimumvalue of pixel value that the display section 68 can display, orresolution which is the minimum value of difference between pixel valuesthat the display section 68 can discriminate.

[0091] Thereafter, processing proceeds to step S12. The pixel valuenormalizing section 66 judges whether or not picture information isstored in the buffer 65. In the case where it is judged at the step S 12that picture information is stored in the buffer 65, processing proceedsto step S13. The pixel value normalizing section 66 reads out pictureinformation of one frame from the buffer 65. Thus, processing proceedsto step S14.

[0092] At the step S 14, the pixel value normalizing section 66 isoperative so that in the case where picture information which has beenread out from the buffer 65 is corrected pixel value, it normalizes itscorrected pixel value in a manner as described above to thereby allowsuch corrected pixel value to be normalized pixel value to deliver it tothe driver 67. Thus, processing returns to the step S12. Moreover, thepixel value normalizing section 66 is operative so that in the casewhere picture information which has been read out from the buffer 65 ispixel value and exposure time, pixel value is corrected by exposure timeto thereby allow that pixel value to be corrected pixel value. Further,the pixel value normalizing section 66 normalizes its corrected pixelvalue in a manner as described above to thereby allow such correctedpixel value to be normalized pixel value to deliver it to the driver 67.Thus, the processing returns to the step S12. Thus, at the driver 67,the display section 68 is driven in accordance with the normalized pixelvalue from the pixel value normalizing section 66. Thus, correspondingpicture image, i.e., picture image effectively utilizing dynamic range(resolution) of the display section 68 is displayed.

[0093] It is to be noted that while the number of bits K that thedisplay section 68 can display as display accuracy of the displaysection 68 is caused to be smaller than the number of bits K′ ofcorrected pixel value in this example, in the case where the number ofbits K which is display accuracy of the display section 68 is more thanthe number of bits K′ of corrected pixel value, there is no necessity tocarry out normalization as described above at the pixel valuenormalizing section 66 and accordingly, the pixel value normalizingsection 66 delivers corrected pixel value to the driver 67 as it is.

[0094] On the other hand, in the case where it is judged at the step S12that picture information is not stored in the buffer 65, processing iscompleted.

[0095]FIG. 11 shows an example of the configuration of a thirdembodiment of digital video camera 101 of FIG. 1. It is to be noted thatthe same reference numerals are respectively attached to portionscorresponding to the case in FIG. 2 or FIG. 3 in the figure, and theirexplanation will be omitted as occasion demands. Namely, the digitalvideo camera 101 of FIG. 11 is constituted essentially similarly to thecase in FIG. 2 or FIG. 3 except that a memory controller 31 and memories32 ₁, 32 ₂, . . . , 32 _(N) are newly provided and a controller 33 isprovided in place of the controller 5.

[0096] It is to be noted that while shutter 2 is constituted with DMDsimilarly to the case in FIG. 2 in the embodiment of FIG. 11, it may beconstituted with liquid crystal shutter as in the case in FIG. 3. InFIG. 11, the shutter 2 is only required to have ability to equally turnON/OFF incidence of light upon the CCD 3 with respect to all pixelsconstituting the CCD 3. Accordingly, there is no necessity to constituteDMD or liquid crystal shutter, etc. which controls, in pixel units,incidence of light upon the CCD 3.

[0097] The memory controller 31 delivers pixel value delivered throughthe A/D converter 4 from the CCD 3 to any one of frame memories 32 ₁ to32 _(N) in accordance with control from the controller 33 to allow it tostore the pixel value. The memories 32 ₁ to 32 _(N) are adapted to storepixel value delivered from the memory controller 31.

[0098] The controller 33 sets plural exposure times in the shutter 2 tocontrol the shutter 2 by respective plural exposure times so that lightfrom object is incident upon the CCD 3. Accordingly, in this case, inthe CCD 3, pixel values constituting one frame are outputted withrespect to respective plural exposure times set at the controller 33.Namely, rays of light by respective plural exposure times that thecontroller 33 has set are incident, within period, upon the CCD 3. Thus,at the CCD 3, with respect to respective frames, pixel values of pluralpictures corresponding to respective plural exposure times areoutputted. Further, the controller 33 controls the memory controller 31in a manner as described above so that pixels constituting pluralpictures corresponding to respective plural exposure times are storedinto the same memory 32 _(n) (n=1, 2, . . . , N) every exposure times.For example, in the controller 33, when N exposure times are assumed tobe set and exposure time short at the n-th order is assumed to be then-th exposure time, the controller 33 controls the memory controller 31so that pixel value of picture corresponding to the n-th exposure timeoutputted through the A/D converter 4 from the CCD 3 is stored into thememory 32 _(n). Moreover, the controller 33 selects one pixel value withrespect to pixel of that position from plural pixel values of pixels ofthe same position constituting pictures corresponding to plural exposuretimes stored in respective memories 32 ₁ to 32 _(N) to constitutepicture of one frame by the selected pixel value. Further, similarly tothe controller 5, the controller 33 corrects, as occasion demands, pixelvalues constituting picture of one frame constituted in a manner asdescribed above on the basis of exposure times when those pixel valuesare obtained to output, in units of one frame, for example, picture dataconsisting of the corrected pixel value. In the following description,at the controller 33, N(N is integer equal to 2 or more) exposure timesas plural exposure times are assumed to be set.

[0099]FIG. 12 shows an example of the configuration of the controller 33of FIG. 11. It is to be noted that the same reference numerals arerespectively attached to portions constituted similarly to thecontroller 5 of FIG. 5 in the figure and their explanation will beomitted as occasion demands. A read-out section 41 reads out pixel valueof remarked pixel from any one of memories 32 ₁ to 32 _(N) in accordancewith control of a control section 42 to deliver it to the buffer 21. Thecontrol section 42 controls the pixel value correcting section 22, theread-out section 41 and a reference parameter determining section 43while making reference to evaluation result of pixel value stored in thebuffer 21 by the evaluating section 23 and N exposure times stored inthe memory 44 as occasion demands.

[0100] The reference parameter determining section 43 determinesreference parameter serving as reference in determining N exposure timeson the basis of control from the control section 42. Namely, thereference parameter determining section 43 determines, as referenceparameter, e.g., parameter for determining one exposure time serving asreference and the remaining N−1 exposure times with that exposure timebeing as reference. In this case, as parameter for determining other(N−1) exposure times of the exposure time serving as reference, thereare, e.g., parameters as described below. Namely, in digital videocameras, in general, plural exposure times which can be used are set inadvance. Accordingly, in the case where a certain exposure time iscaused to be reference, exposure time short by one step (stage) withrespect thereto or exposure time long by one step (stage) with respectthereto is univocally determined. Accordingly, such number of steps(stages) may be used as reference parameter.

[0101] When the reference parameter determining section 43 determinesreference parameter, it sets N exposure times on the basis of thereference parameter. Namely, the reference parameter determining section43 allows, e.g., exposure time determined as reference parameter to bethe first exposure time which is the shortest value of N exposure times,and allows exposure times longer every number of stages determined asreference parameter to be set to the second exposure time, the thirdexposure time, . . . , the N-th exposure time in order. Accordingly,when, e.g., plural exposure times set in advance in digital video cameraare expressed as S₁, S₂, . . . , S_(N) (M is integer value larger thanN) in order of shorter exposure times, N exposure times of S_(k),S_(k−1), . . . , S_(k−N+1) are set with respect to the referenceparameter where exposure time of reference is S_(k) (k is integergreater than 1 and smaller than M) and the number of stages is 1.Moreover, with respect to reference parameter where, e.g., exposure timeof reference is S_(k) and the number of stages is 2, N exposure times ofS_(k), S_(k−2), S_(k−4), . . . , S_(k−2(N−1)) are set. It is to be notedthat, at the reference parameter determining section 43, at the time ofsetting N exposure times on the basis of reference parameter, such anapproach may be also employed to linearly or non-linearly change thenumber of stages of those N exposure times adjacent to each other.Namely, at the reference parameter determining section 43, N exposuretimes, e.g., S_(k), S_(k−1), S_(k−3), S_(k−6), S_(k−10), . . . , can beset.

[0102] The memory 44 stores (overwrites) N exposure times set at thereference parameter determining section 43. N exposure times stored inthe memory 44 are adapted so that they are delivered to shutter controlsection 12, memory controller 31 (FIG. 11) and control section 42. Thus,the shutter control section 12 controls the shutter 2 so that light fromobject is incident upon the CCD 3 at respective N exposure times, andthe memory controller 31 allows any one of memories 32 ₁ to 32 _(N) tostore, every exposure time, pixel values from A/D converter 4 which areobtained with respect to N exposure times.

[0103] The operation of the digital video camera of FIG. 11 will bedescribed with reference to the flowchart of FIG. 13.

[0104] First of all, at step S21, the reference parameter determiningsection 43 of the controller 33 (FIG. 12) sets N exposure times on thebasis of reference parameter of default to transmit such exposure timesto the memory 44 to allow the memory 44 to store them. The shuttercontrol section 12 controls the shutter 2 in accordance with respectiveN exposure times stored in the memory 44, i.e., controls the shutter 2in accordance with respective N exposure times in tine divisional mannerwithin frame period. Thus, pixel values constituting picturescorresponding to respective N exposure times are outputted in timedivisional manner from the CCD 3. Pixel values constituting picturescorresponding to respective N exposure times that the CCD 3 outputs intime divisional manner are delivered to the memory controller 31. Thememory controller 31 makes reference to the memory 44 to therebyrecognize respective N exposure times to deliver pixel valueconstituting picture image corresponding to the first exposure time)(the shortest exposure time of the N exposure times) to memory 32 ₁ tostore it at address corresponding to pixel of that pixel value.Similarly, the memory controller 31 respectively also delivers pixelvalues constituting pictures corresponding to the second exposure timeto the N-th exposure time to memories 32 ₂ to 32 _(N) to allow thosememories to store such pixel values. Thus, pixel values obtained atdifferent exposure times with respect to pixels constituting picture ofthe same content are stored in the memories 32 ₁ to 32 _(N).

[0105] Thereafter, processing proceeds to step S22. The control section42 allows pixels constituting picture to be remarked pixel by, e.g.,raster scan order to control the read-out section 41 to thereby allowmemories set at default (default memories) of memories 32 ₁ to 32 _(N)to read out pixel value of remarked pixel stored thereat. It is to benoted that memory caused to serve as default memory is not particularlylimited, but arbitrary memory, e.g., memory 32 _(N/2) or 32 _((N−1)/2),etc. of memories 32 ₁ to 32 _(N) may be default memory. In this example,memories that the read-out section 41 reads out pixel value of memories32 ₁ to 32 _(N) will be referred to as remarked memory as occasiondemands. When the read-out section 41 reads out pixel value of remarkedpixel from the remarked memory in accordance with control of the controlsection 42, it delivers its pixel value to the buffer 21 to allow thebuffer 21 to store the pixel value. Thus, processing proceeds to stepS23.

[0106] At step S23, the evaluating section 23 evaluates pixel value ofremarked pixel stored in the buffer 21 to output its evaluation resultto the control section 42. Thus, processing proceeds to step S24.

[0107] At the step S24, the control section 42 judges on the basis ofevaluation result from the evaluating section 23 whether or not pixelvalue of remarked pixel is in white overexposure state (and whether ornot movement of remarked pixel is large as occasion demands). In thestate where it is judged at the step S24 that pixel value of theremarked pixel is in white overexposure state (or in the case where itis judged that movement is large), i.e., in the case where exposure timeused when pixel value which has been read out from the remarked memoryis too long, processing proceeds to step S25. The control section 42judges whether or not the remarked memory is memory in which pixel valueof picture corresponding to the shortest exposure time (the firstexposure time) (hereinafter referred to as shortest memory as occasiondemands as occasion demands) (memory 32 ₁ in this embodiment).

[0108] In the case where it is judged at the step S25 that the remarkedmemory is not the shortest memory, processing proceeds to step S26. Thecontrol section 42 controls the read-out section 41 to thereby changethe remarked memory to memory in which pixel value of picturecorresponding to exposure time shorter next is stored. Namely, in thisembodiment, when the remarked memory is assumed to be memory 32 _(n),the control section 42 changes the remarked memory from memory 32 _(n)to memory 32 _(n−1). Further, the control section 42 controls theread-out section 41 in a manner to read out pixel value of the remarkedpixel from the changed remarked memory. Thus, processing returns to thestep S23 and similar processing will be repeated at times subsequentthereto. Moreover, it is judged at the step S25 that the remarked memoryis the shortest memory, i.e., in the case where even when there isemployed pixel value obtained by using the shortest exposure time of Nexposure times set now, there is in white overexposure state, and thereis thus necessity to shorten exposure time in order to avoid whileoverflow state, processing proceeds to step S27. The control section 42delivers request for allowing the entirety of N exposure times or aportion thereof (e.g., several shorter exposure times of N exposuretimes) to the reference parameter determining section 43. Thus,processing proceeds to step S28.

[0109] On the other hand, in the case where it is judged at the step S24that pixel value of remarked pixel is not in the white overexposurestate, processing proceeds to step S29. The control section 42 judges onthe basis of evaluation result from the evaluating section unit 23whether or not pixel value of remarked pixel is in the state of blackunderexposure. In the case where it is judged at the step 29 that pixelvalue of the remarked pixel is in the state of black underexposure,i.e., in the case where exposure time used when pixel value which hasbeen read out from the remarked memory is obtained is too short,processing proceeds to step S30. The control section 42 judges whetheror not the remarked memory is memory in which pixel value of picturecorresponding to the longest exposure time (the N-th exposure time inthis embodiment) (hereinafter referred to as longest memory) (memory 32_(N) in this embodiment).

[0110] In the case where it is judged at the step S30 that the remarkedmemory is not the longest memory, processing proceeds to step S31. Thecontrol section 42 controls the read-out section 41 to thereby changethe remarked memory to memory in which pixel value of picturecorresponding to exposure time longer next is stored. Namely, in thisembodiment, when the remarked memory is memory 32 _(n), the controlsection 42 changes the remarked memory from memory 32 _(n) to memory 32_(n+1). Further, the control section 42 controls the read-out section 41so as to read out pixel value of remarked pixel from the changedremarked memory. Thus, processing returns to the step S23, and similarprocessing will be repeated at times subsequent thereto. Moreover, inthe case where it is judged at the step S30 that the remarked memory isthe longest memory, i.e., in the case where even when there is employedpixel value obtained by using the longest exposure time of N exposuretimes set now, there is in the state of black underexposure, and thereis thus necessity to allow the exposure time to be longer in order toavoid the state of black underexposure, processing proceeds to step S32.The control section 42 delivers a request for allowing the entirety of Nexposure times or a portion thereof(e.g., several longer exposure timesof N exposure times) to be longer to the reference parameter determiningsection 43. Thus, processing proceeds to step S28.

[0111] On the other hand, in the case where it is judged at the step S29that pixel value of remarked pixel is not in the state of blackunderexposure, i.e., in the case where pixel value of the remarked pixelis not in either state of white overexposure and black underexposure,processing proceeds to the step S28. Thus, pixel value of remarked pixelstored in the buffer 21 is delivered to the pixel value correctingsection 22. Moreover, at the step S28, the control section 42 recognizesexposure time when pixel value stored in the buffer 21 is obtained bymaking reference to memory 44 to deliver its exposure time to the pixelvalue correcting section 22. Further, at the step S28, the pixel valuecorrecting section 22 allows pixel value of the remarked pixel from thebuffer 21 and exposure time used for obtaining that pixel value from thecontrol section 42 to correspond to each other to deliver the pixelvalue and the exposure time to the memory 6 to allow the memory 6 tostore them. Accordingly, at the step S28, at the pixel value correctingsection 22, in principle, pixel values which are not in either state ofwhite overexposure and black underexposure of plural pixel values storedin memories 32 ₁ to 32 _(N) with respect to remarked pixel are selectedand are stored into the memory 6. It is to be noted that in the casewhere there does not exist pixel value placed in the state of whiteoverexposure or black underexposure of plural pixel values stored inmemories 32 ₁ to 32 _(N) with respect to the remarked pixel, pixel valuein which the degree of the state of white overexposure or blackunderexposure is the lowest is selected and is stored into the memory 6,and a request for change of exposure time is made from the controlsection 42 to the reference parameter determining section 43 in order tocancel the state of that white overexposure or black underexposure.

[0112] After pixel value and exposure time of remarked pixel is causedto be stored into the memory 6, processing proceeds to step S33, atwhich whether or not all of pixel values constituting picture of oneframe are written into the memory 6 is judged. In the case where it isjudged at the step S33 that all of pixel values constituting picture ofone frame are not yet written into the memory 6, processing proceeds tostep S34. Pixel next to remarked pixel now in order of raster scan iscaused to be newly remarked pixel. At the read-out section 41, pixelvalue of that remarked pixel is read out from the remarked memory.Further, processing returns to the step S23, and similar operation willbe repeated at times subsequent thereto. Moreover, in the case where itis judged at the step S33 that all of pixels constituting one frame arewritten into the memory 6, i.e., in the case where pixel values of allpixels constituting one frame and exposure times caused to correspondthereto are stored, processing proceeds to step S35. Thus, the pixelvalue collecting section 22 reads out respective pixel values from thememory 6 similarly to the case at the step S8 of FIG. 7 to correctrespective pixel values on the basis of exposure times caused tocorrespond to those pixel values to output picture data of one frameconstituted with the corrected pixel value.

[0113] Further, processing proceeds to step S36. The reference parameterdetermining section 43 is operative so that in the case where there isrequest for allowing exposure time to be shorter or longer of exposuretime at step S27 or S32, it determines reference parameter for a secondtime so that exposure time in accordance with that request is set.Further, the reference parameter determining section 43 sets N exposuretimes for a second time on the basis of the reference parameter whichhas been determined for a second time. Thus, processing proceeds to stepS37. It is to be noted that in the case where there is no request forallowing exposure time to be shorter or longer, the reference parameterdetermining section 43 sets the same N exposure times as those atprevious time by using the reference parameter determined last as it is.

[0114] At the step S37, the reference parameter determining section 43delivers N exposure times set at the step S36 to memory 44 to allow thememory to store them. Thus, processing returns to the step S22, andsimilar processing will be repeated with respect to the next frame attimes subsequent thereto.

[0115] As stated above, such an approach is employed to set pluralexposure times to obtain picture images corresponding to respectiveplural exposure times to select pixel values which are not placed in thestate of white overexposure and black underexposure to therebyconstitute picture images of respective frames. Accordingly, even whenthere is employed object of high contrast, it is possible to obtainpicture of which detail is not injured. Further, also in this case,similarly to the case in FIG. 2 and FIG. 3, it is possible to providethe same effect as in the case where dynamic range of CCD 3 (or A/Dconverter 4) is broadened.

[0116] It is to be noted that, also in the embodiment of FIG. 11,respective pixel values stored in the memory 6 may be outputted alongwith exposure times caused to correspond to those pixel values as theyare and may be recorded onto recording medium 103 or may be caused toundergo transmission through the transmission medium 104.

[0117] While, in the above-described case, at the pixel value correctingsection 22, correction of pixel value by exposure time is carried out onthe premise that exposure time and pixel value have proportionalrelationship, in the case where exposure time and pixel value have notproportional relationship, there is included in corrected pixel valueobtained as the result of correction of pixel value with theproportional relationship being as the premise.

[0118] In view of the above, the pixel value correcting section 22 mayestimate the relationship between exposure time and pixel value to carryout correction of pixel value by exposure time on the basis of theestimation result. Namely, in tile case where, e.g., output of A/Dconverter 4 is 8 bits, the pixel value correcting section 22 variesexposure time as shown in FIG. 14 with respect to several brightnessesto obtain pixel value in the range of 0 to 255 (=2⁸−1). In FIG. 14, markx indicates pixel values obtained with respect to respective exposuretimes. Further, the pixel value connecting section 22 determinesapproximate curve which approximates the relationship between exposuretime and pixel value every respective brightnesses by using pixel valuesby respective exposure tines obtained with respect to respectivebrightnesses. In the embodment of FIG. 14, approximate curves L11, L12,L13 indicating the relationship between exposure time and pixel valuewith respect to three brightnesses are obtained. Further, the pixelvalue correcting section 22 carries out correction of pixel value byexposure time on the basis of the approximate curves thus obtained.

[0119] Namely, as shown in FIG. 15, the pixel value correcting section22 determines approximate curve closest to the point (p₀, t₀) (indicatedby mark x in FIG. 15) prescribed by pixel value p₀ to be corrected andexposure time t₀ with respect to that pixel value. In the embodiment ofFIG. 15, approximate curve L21 is curve closest to the point (p₀, t₀).Further, in the case where pixel value p₀ with respect to exposure timet₀ is corrected to pixel value with respect to exposure tine t₁, thepixel value correcting section 22 determines the point corresponding toexposure time t₁ (indicated by mark ◯ in FIG. 15) on the approximatecurve L21 to allow pixel value p₁ indicated by that point to becorrected pixel value. In this case, corrected pixel value having goodaccuracy can be obtained.

[0120] Namely, in the case where exposure time and pixel value have notproportional relationship, but has non-linear relationship as indicatedby approximate curve L21 of FIG. 15, for example, when correction ofpixel value p₀ is carried out with the proportional relationship betweenexposure time and pixel value being as the premise, pixel value p₀ iscorrected to pixel value p₃ indicated by the point corresponding toexposure time t₁ (indicated by mark □ in FIG. 15) on the straight lineL22 passing through the origin. As a result, error ∈(=p₃-p₁) isproduced. On the contrary, in the case where approximate curve is usedto carry out correction of pixel value, such an error does not takeplace.

[0121] Meanwhile, in accordance with the processing by the flowchart ofFIG. 13, when pixel value which is not placed in either state of whiteoverexposure and black underexposure of plural pixel values of remarkedpixels obtained with respect to respective N exposure times stored in Nmemories 32 ₁ to 32 _(N) is first found, the pixel value correctingsection 22 selects, as pixel value of remarked pixel, the pixel valuewhich has been first found to allow the memory 6 to store that pixelvalue.

[0122] However, the pixel value correcting section 22 may be adapted toselect, as pixel value of remarked pixel, pixel value close to areference pixel value in the state where, e.g., a certain pixel value iscaused to be the reference pixel value. Namely, in general, it isconsidered that sensitivity of the CCD 3 is the highest at value ofintermediate degree of the range of pixel value that the A/D converter 4outputs. In view of the above, reference pixel value is caused to bevalue of intermediate degree of the range of pixel value that the A/Dconverter 4 outputs, and the pixel value correcting section 22 may beadapted to select, as pixel value of remarked pixel, pixel value closestto reference pixel value of N pixel values of remarked pixel stored inrespective memories 32 ₁ to 32 _(N).

[0123] In accordance with pixel value selected in this way, pictureimage as described below is constructed. Namely, FIG. 16 shows pixelvalue on a certain horizontal line of picture image obtained by usingplural exposure times with respect to a certain object. In theembodiment of FIG. 16, as plural exposure times, there are used sevenexposure times of ⅕, {fraction (1/10)}, {fraction (1/20)}, {fraction(1/30)}, {fraction (1/60)}, {fraction (1/120)}, {fraction (1/180)} sec.In addition, in FIG. 16, curves L31, L32, L33, L34, L35, L36, L37indicate pixel values (outputs of A/D converter 4) obtained byrespective exposure times of ⅕, {fraction (1/10)}, {fraction (1/20)},{fraction (1/30)}, {fraction (1/60)}, {fraction (1/120)}, {fraction(1/180)} sec.

[0124]FIG. 17 shows corrected pixel value obtained by correcting pixelvalue of FIG. 16 by exposure time. It is assumed in the embodiment ofFIG. 17 that pixel value and exposure time have proportionalrelationship, and {fraction (1/10)} sec. is caused to be exposure timeof reference and corrected pixel value is determined by multiplyingpixel value where exposure time is 1/S [sec.] by S/10. Moreover, in theembodiment of FIG. 17, there are shown corrected pixel values withrespect to five exposure times where exposure times are {fraction(1/10)}, {fraction (1/20)}, {fraction (1/30)}, {fraction (1/60)},{fraction (1/120)}, and curves L41, L42, L43, L44, L45 show correctedpixel values of pixel values obtained by respective exposure times of{fraction (1/10)}, {fraction (1/20)}, {fraction (1/30)}, {fraction(1/60)}, {fraction (1/120)} sec.

[0125] Since pixel value of FIG. 16 is output of AID converter, dynamicrange of 8 bits is obtained, whereas since pixel value of FIG. 17 iscorrected pixel value, dynamic range greater than 8 bits is obtained.

[0126]FIG. 18 shows picture image constituted by selecting collectedpixel value of pixel value closest to the reference pixel value. In theembodiment of FIG. 18, 100 is used as reference pixel value.Accordingly, when pixel value obtained by exposure time T is assumed asP_(T) and collected pixel value obtained by correcting pixel value P_(T)is assumed as f(P_(T)), picture image of FIG. 18 is constituted by,e.g., corrected pixel value P obtained by the formulaP=f(min(P_(T)−P_(B))²). In the above formula, min (P_(T)−P_(B))²represents P_(T) which minimizes (P_(T)−P_(B))². In addition, P_(B)represents reference pixel value and is equal to 100 in this case asdescribed above.

[0127] By making comparison between FIG. 16 and FIG. 18, in the casewhere output of the A/D converter 4 is used as it is, only picture image(FIG. 16) in the range where pixel value is 0 to 255 can be obtained. Onthe other hand, in the case by the corrected pixel value, picture imageby pixel value in broader range (about 0 to 700 in FIG. 18), i.e.,picture image having broad dynamic range can be obtained.

[0128] The above-described series of processing may be carried out byhandware, or may be carried out by software. In the case where a seriesof processing are carried out by software, program constituting thatsoftware is installed into widely used computer, etc.

[0129]FIG. 19 shows an example of the configuration of an embodiment ofcomputer into which program which executes the above-described series ofprocessing is installed. Program can be recorded in advance with respectto a hard disc 205 or a ROM 203 as recording medium included within thecomputer. Alternatively, program may be temporarily or permernentlystored (recorded) on a removable recording medium 211 such as floppydisc, CD-ROM (Compact Disc Read Only Memory), MO (Magneto optical) disc,DVD (Digital Versatile Disc), magnetic disc, or semiconductor memory,etc. Such removable recording medium 211 can be provided as theso-called package software. In this case, program is installed from theremovable recording medium 211 as described above into the computer. Inaddition, program may be transferred by wireless to the computer throughartificial satellite for digital satellite broadcast from down loadsite, or may be transferred to the computer by wire through network suchas LAN (Local Area Network) or internet. In the computer, programtransferred in such a way may be received at a communication section 208and installed into hard disc 205 included.

[0130] The computer includes a CPU (Central Processing Unit) 202. Aninput/output interface 210 is connected to the CPU 202 through a bus201. When command is inputted as the result of the fact that an inputsection 207 comprised of keyboard, mouse or microphone, etc. is operatedor is caused to undergo similar operation by user through aninput/output interface 210, the CPU 202 executes program stored in a ROM(Read Only Memory) 203 in accordance with the command. Alternatively,the CPU 202 loads, into a RAM (Random Access Memory) 204, program storedon the hard disc 205, program transferred from the satellite or thenetwork, received at the communication section 208 and installed on thehard disc 205, or program read out from the removable recording medium211 fitted at a drive 209 and installed on the hard disc 205, andexecutes such program. Thus, the CPU 202 carries out processing inaccordance with the above-described flowchart, or processing carried outby the configuration of the above-described block diagrams. In addition,the CPU 202 outputs, e.g., as occasion demands its processing resultfrom an output section constituted with LCD (Liquid Crystal Display) orspeaker, etc. through the input/output interface 210, transmits it fromthe communication section 208 and records it onto the hard disc 205,etc.

[0131] It is to be noted that, in this specification, it is notnecessarily required to process processing steps which describe programfor allowing the computer to carry out various processing in a timedivisional manner along the order described as the flowchart, and suchprocessing steps include processing executed in parallel or individually(e.g., parallel processing or processing by object). Moreover, programmay be processed by single computer, or may be caused to undergodistribution processing by plural computers. Further, program may betransferred to remote computer, at which it is executed.

[0132] It is to be noted that while shutter capable of controllingexposure every respective pixels of CCD 3 is used as shutter 2 in theembodiments of FIGS. 2 to 4, there may be used, e.g., shutter capable ofcontrolling exposure every plural pixels such as two pixels of CCD 3 inaddition to the above as shutter 2. Further, this invention can beapplied to both moving picture and still picture.

[0133] Furthermore, while, in this embodiment, particularly referencehas not been made to stop (iris), such an approach may be also employedto carry out control of stop on the basis of evaluation result of pixelvalue. Namely, in the case where there is the state of blackunderexposure or white over exposure, such an approach may be employedto carry out control so as to open stop or close stop. In addition, auser may manually adjust stop.

INDUSTRIAL APPLICABILITY

[0134] In accordance with the first image pick-up apparatus and thefirst image pick-up method, and the program and the program recordingmedium of this invention, pixel values acquired from image pick-up meanshaving light receiving surface for receiving light from object to carryout photo-electric conversion and adapted to output pixel value obtainedas the result of the photo-electric conversion are evaluated, andexposure time with respect to light receiving surface is controlled inpixel units on the basis of the evaluation result. Accordingly, alsowith respect to object of high contrast, it is possible to obtainpicture image of which detail is not injured.

[0135] In accordance with the data structure and the data recordingmedium of this invention, plural pixel values that the image pick-upapparatus for picking up image of object outputs and exposure timesevery respective pixel values used in the image pickup apparatus forobtaining respective plural pixel values are caused to correspond toeach other. Accordingly, pixel values are corrected on the basis ofexposure time, thereby making it possible to obtain picture image inwhich constant exposure is used with respect to the entirety and havingbroad dynamic range.

[0136] In accordance with the image pick-up control apparatus of thisinvention, pixel values are evaluated and control signal forcontrolling, in predetermined surface units, exposure time with respectto light receiving surface is outputted to the image pick-up unit on thebasis of the evaluation result. Accordingly, also with respect to objectof high contrast, it is possible to obtain picture image of which detailis not injured.

[0137] In accordance with the second image pick-up apparatus and thesecond image pick-tip method of this invention, plural exposure timeswith respect to light receiving surface at image pick-up means havinglight receiving surface for receiving light from object to carry outphoto-electric conversion and adapted to output pixel value obtained asthe result of the photo-electric conversion are controlled, and onepixel value is selected from pixel values corresponding to respectiveplural exposure times of respective pixel positions which are obtainedby picking up image of object by plural exposure times on the basis ofthat control. Accordingly, also with respect to object of high contrast,it is possible to obtain picture image of which detail is not injured.

[0138] In accordance with the second program and the second programrecording medium of this invention, pixel values corresponding torespective plural exposure times of respective pixel positions which areobtained by controlling plural exposure times with respect to lightreceiving surface at the image pick-up unit having light receivingsurface for receiving light from object to carry out photo-electricconversion and adapted to output pixel value obtained as the result ofthe photo-electric conversion are evaluated, and one pixel value isselected from pixel values corresponding to respective plural exposuretimes on the basis of the evaluation result. Accordingly, also withrespect to object of high contrast, it is possible to obtain pictureimage of which detail is not injured.

1. An image pick-up apparatus for picking up an image of object, theimage pickup apparatus comprising: image pick-up means having a lightreceiving surface for receiving light from the object to carry outphoto-electric conversion, and adapted to output pixel value obtained asthe result of the photo-electric conversion; evaluating means forevaluating the pixel value; and control means for controlling, in pixelunits, exposure time with respect to the light receiving surface on thebasis of evaluation result by the evaluating means.
 2. The image pick-upapparatus as set forth in claim 1, wherein the evaluating meansevaluates whether or not the pixel value is value within a predeterminedrange; and wherein when the pixel value is not value within thepredetermined range, the control means controls exposure time withrespect to pixel of the light receiving surface corresponding to thatpixel value so that the pixel value is caused to be value within thepredetermined range.
 3. The image pick-up apparatus as set forth inclaim 2, wherein the control means is operative so that when the pixelvalue is a predetermined value or more, it shortens exposure time withrespect to pixel of the light receiving surface corresponding to thatpixel value.
 4. The image pick-up apparatus as set forth in claim 2,wherein the control means is operative so that when the pixel value isless than a predetermined value, it elongates exposure time with respectto pixel of the light receiving surface corresponding to that pixelvalue.
 5. The image pick-up apparatus as set forth in claim 2, whereinthe control means comprises: plural reflection mirrors rotatablyprovided for reflecting the light onto the light receiving surface; anda rotational control section for controlling rotations of the respectiveplural reflection minors on the basis of evaluation result by theevaluating means to thereby control exposure time with respect to thelight receiving surface in pixel units.
 6. The image pick-up apparatusas set forth in claim 2, wherein the control means comprises: a liquidcrystal shutter for transmitting or reflecting the light to allow thelight to be incident upon the light receiving surface; and a liquidcrystal control section for controlling, in pixel units, transmission orreflection of light in the liquid crystal shutter on the basis ofevaluation result by the evaluating means to thereby control exposuretime with respect to the light receiving surface in pixel units.
 7. Theimage pick-up apparatus as set forth in claim 2, wherein the evaluatingmeans comprises: memory means for storing the pixel value that the imagepick-up means outputs; and a movement judging section for comparingremarked pixel value to be remarked of the pixel value that the imagepick-up means outputs and pixel value corresponding to the remarkedpixel value of pixel values stored in the memory means to thereby judgemovement of remarked pixel which is pixel corresponding to the remarkedpixel value; and wherein the control means controls exposure time withrespect to the remarked pixel of the light receiving surface on thebasis of judgment result of movement of the remarked pixel value.
 8. Theimage pick-up apparatus as set forth in claim 1, which further comprisescorrecting means for correcting the pixel value that the image pick-upmeans outputs on the basis of exposure time of pixel corresponding tothat pixel value.
 9. The image pick-up apparatus as set forth in claim8, which further comprises memory means for storing plural pixel valuesthat the image pick-up means outputs and exposure times of respectivepixels corresponding to those plural pixel values, wherein thecorrecting means is operative so that when the longest time of theplural exposure times stored in the memory means is assumed to be1/S_(BASE) and exposure time of pixel value stored in the memory meansis assumed to be 1/S, it multiplies the pixel value stored in the memorymeans by S/S_(BASE) to thereby correct the pixel value.
 10. The imagepick-up apparatus as set forth in claim 8, which further comprises adisplay control section for displaying picture image on a displaysection for displaying picture image in accordance with pixel valuecorrected at the correcting means.
 11. The image pick-up apparatus asset forth in claim 1, which further comprises a display control sectionfor displaying picture image on a display section for displaying pictureimage in accordance with pixel value that the image pick-up meansoutputs and exposure time of pixel corresponding to that pixel value.12. The image pick-up apparatus as set forth in claim 11, wherein thedisplay control section comprises: memory means for storing plural pixelvalues that the image pick-up means outputs and exposure times of pixelscorresponding to the respective pixel values; a correcting section forcorrecting respective plural pixel values stored in the memory means byexposure times corresponding thereto; and a normalizing section fornormalizing the corrected pixel value in accordance with displayaccuracy of the display section.
 13. The image pick-up apparatus as setforth in claim 1, which further comprises memory means for storingplural pixel values that the image pick-up means outputs and exposuretimes of pixels corresponding to the respective pixel values in such amanner to correspond to each other.
 14. An image pick-up method ofpicking up image of object, the image pick-up method comprising: anevaluation step of evaluating pixel value acquired from an image pick-upsection having a light receiving surface for receiving light from theobject to carry out photo-electric conversion and adapted to output thepixel value obtained as the result of the photo-electric conversion; anda control step of controlling, in pixel units, exposure time withrespect to the light receiving surface on the basis of evaluation resultby the evaluation step.
 15. A program for allowing computer to carry outimage pick-up processing for picking up image of object, the programincluding: an evaluation step of evaluating pixel value acquired from animage pick-up section having a light receiving surface for receivinglight from the object to carry out photo-electric conversion and adaptedto output the pixel value obtained as the result of the photo-electricconversion; and a control step of controlling, in pixel units, exposuretime with respect to the light receiving surface on the basis ofevaluation result by the evaluation step.
 16. A program recording mediumadapted so that program for allowing computer to carry out image pick-upprocessing which picks up image of object is recorded, wherein there isrecorded program comprising: an evaluation step of evaluating pixelvalue acquired from an image pick-up section having a light receivingsurface for receiving light from the object and adapted to output pixelvalue obtained as the result of the photo-electric conversion; and acontrol step of controlling, in pixel units, exposure time with respectto the light receiving surface on the basis of evaluation result by theevaluation step.
 17. A data structure of data which can be read bycomputer, wherein plural pixel values that an image pick-up unit forpicking up image of object outputs and exposure times every respectivepixel values used in the image pick-up unit in order to obtainrespective ones of the plural pixel values are caused to correspond toeach other.
 18. A data recording medium adapted so that data which canbe read by computer is recorded, wherein plural pixel values that animage pick-up unit for picking up image of object outputs and exposuretimes every respective pixels used in the image pick-up unit in order toobtain respective ones of the plural pixel values are recorded in such amanner that they are caused to correspond to each other.
 19. An imagepick-up control apparatus for controlling an image pick-up sectionhaving a light receiving surface for receiving light from object andadapted to output pixel value obtained as the result of thephoto-electric conversion, the image pick-up control apparatuscomprising: an evaluating section for evaluating the pixel value; andcontrol means for outputting, to the image pick-up section, a controlsignal for controlling, in a predetermined surface unit, exposure timewith respect to the light receiving surface on the basis of evaluationresult by the evaluating section.
 20. An image pick-up apparatus forpicking up image of object, the image pick-up apparatus comprising:image pick-up means having a light receiving surface for receiving lightfrom the object to carry out photo-electric conversion and adapted tooutput pixel value obtained as the result of the photo-electricconversion; control means for controlling plural exposure times withrespect to the light receiving surface; and selector means for selectingone pixel value from pixel values corresponding to respective ones ofthe plural exposure times of respective pixel positions which areobtained by picking up image of the object by the plural exposure timeson the basis of control of the control means in the image pick-up means.21. The image pick-up apparatus as set forth in claim 20, wherein theselector means selects one pixel value closest to a predetermined valuefrom pixel values corresponding to respective ones of the pluralexposure times.
 22. The image pick-up apparatus as set forth in claim20, which further comprises evaluating means for evaluating at least onepixel value of pixel values corresponding to respective ones of theplural exposure times, wherein the control means changes at least one ofthe plural exposure times on the basis of evaluation result by theevaluating means.
 23. The image pick-up apparatus as set forth in claim22, wherein the evaluating means evaluates whether or not at least onepixel value of pixel values corresponding to the plural exposure timesis value within a predetermined range; and wherein the selector meansselects pixel value corresponding to exposure time except for exposuretime corresponding to the at least one pixel value of the pluralexposure times when the at least one pixel value is not value within apredetermined range.
 24. The image pick-up apparatus as set forth inclaim 23, wherein the selector means selects pixel value correspondingto exposure time shorter than exposure time corresponding to the atleast one pixel value of the plural exposure times when the at least onepixel value is predetermined value or more.
 25. The image pick-upapparatus as set forth in claim 23, wherein the selector means selectspixel value corresponding to exposure time longer than exposure timecorresponding to the at least one pixel value of the plural exposuretimes when the at least one pixel value is less than the predeterminedvalue.
 26. The image pick-up apparatus as set forth in claim 22, whereinthe control means comprises: plural reflection mirrors rotatablyprovided for reflecting the light onto the light receiving surface; anda rotational control section for controlling rotations of the respectiveplural reflection mirrors on the basis of evaluation result by theevaluating means to thereby control exposure time with respect to thelight receiving surface.
 27. The image pick-up apparatus as set forth inclaim 22, wherein the control means comprises: a liquid crystal shutterfor transmitting or reflecting the light to allow the light to beincident upon the light receiving surface; and a liquid crystal controlsection for controlling, in units of the light receiving surface,transmission or reflection of light in the liquid crystal shutter on thebasis of evaluation result by the evaluating means to thereby controlexposure time with respect to the light receiving surface.
 28. The imagepick-up apparatus as set forth in claim 20, which further comprisescorrecting means for correcting the pixel values of respective pixelpositions that the image pick-up means outputs on the basis of exposuretimes corresponding to those pixel values.
 29. The image pick-upapparatus as set forth in claim 28, which further comprises memory meansfor storing pixel value selected at the selector means.
 30. The imagepick-up apparatus as set forth in claim 29, wherein the evaluating meanscomprises a movement judging section for comparing remarked pixel valueto be remarked of pixel values that the image pick-up means outputs andpixel value corresponding to the remarked pixel value of pixel valuesstored in the memory means to thereby judge movement of remarked pixelwhich is pixel with respect to the remarked pixel value; and wherein thecontrol means controls the exposure time on the basis of judgment resultof movement of the remarked pixel.
 31. The image pick-up apparatus asset forth in claim 29, wherein the memory means stores pixel values ofrespective pixel positions selected at the selector means, and storesexposure times of those pixel values.
 32. The image pick-up apparatus asset forth in claim 31, wherein the correcting means is operative so thatwhen the longest exposure time of plural exposure times stored in thememory means is assumed to be 1/S_(BASE) and exposure time of pixelvalue stored in the memory means is assumed to be 1/S, it multipliespixel value stored in the memory means by S/S_(BASE) to thereby correctthe pixel value.
 33. The image pick-tip apparatus as set forth in claim29, which further comprises a display control section for displayingpicture image on a display section for displaying picture image inaccordance with pixel value corrected at the correcting means.
 34. Theimage pick-up apparatus as set forth in claim 20, which furthercomprises a display control section for displaying picture image on adisplay section for displaying picture image in accordance with pixelvalue selected at the selector means and exposure time corresponding tothat pixel value.
 35. The image pick-up apparatus as set forth in claim34, wherein the display control section comprises: a correcting sectionfor correcting respective ones of plural pixel values stored in thememory means by exposure times corresponding thereto; and a normalizingsection for normalizing the corrected pixel value in accordance withdisplay accuracy of the display section.
 36. An image pick-up method ofpicking up image of object, the image pick-up method comprises: acontrol step of controlling plural exposure times with respect to alight receiving surface at an image pick-up section having a lightreceiving surface for receiving light from the object to carry outphoto-electric conversion and adapted to output pixel value obtained asthe result of the photo-electric conversion; and a selection step ofselecting one pixel value from pixel values corresponding to respectiveones of the plural exposure times of respective pixel positions whichare obtained by picking up image of the object by the plural exposuretimes on the basis of control at the control step in the image pick-upsection.
 37. A program for allowing computer to carry out image pick-upprocessing for picking up image of object, the program comprising: anevaluation step of evaluating pixel values corresponding to respectiveones of the plural exposure times of respective pixel positions whichare obtained by controlling plural exposure times with respect to alight receiving surface in an image pick-up section having the lightreceiving surface for receiving light from the object to carry outphoto-electric conversion and adapted to output pixel value obtained asthe result of the photo-electric conversion; and a selection step ofselecting one pixel value from pixel values corresponding to respectiveones of the plural exposure times on the basis of evaluation result bythe evaluation step.
 38. A program recording medium adapted so thatprogram for allowing computer to carry out image pick-up processing forpicking up image of object is recorded, wherein there is recordedprogram comprising: an evaluation step of evaluating pixel valuescorresponding to respective ones of the plural exposure times ofrespective pixel positions which are obtained by controlling pluralexposure times with respect to a light receiving surface in an imagepick-up section having the light receiving surface for receiving lightfrom the object to carry out photo-electric conversion and adapted tooutput pixel value obtained as the result of the photo-electricconversion; and a selection step of selecting one pixel value from pixelvalues corresponding to respective ones of the plural exposure times onthe basis of evaluation result by the evaluation step.